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Abstract. The u-homology formulas for unitarizable modules at negative levels 
over classical Lie algebras of infinite rank of types gl{n), sp(2n) and so(2n) are ob- 
tained. As a consequence, we recover the Enright's formulas for three Hermitian 
symmetric pairs of classical types (SU{p, q), SU(p) x SU{q)), (Sp(2n),U{n)) and 
{SO*{2n),U{n)). 



1. Introduction 

In analogy to Kostant's u-cohomology formulas [Ko], Enright establishes similar for- 
mulas [E] for unitarizable highest weight modules of Hermitian symmetric pairs in term 
of certain complicated subsets of the Weyl groups. The argument there is intricate and 
involves several equivalences of categories and non-trivial combinatorics of the Weyl 
groups. Kostant's formula can be rephrased by saying the Kazhdan-Lusztig polyno- 
mials associated to finite-dimensional module are monomials. The same statement is 
true by Enright's formulas for unitarizable highest weight modules. Except for the re- 
semblance of the formulas, there was no obvious connection between Enright's formula 
and Kostant's formula. 

However, the modules appearing in the Howe duality at negative levels [W, HI, H2] 
over classical Lie algebras of infinite rank are unitarizable modules (cf . [EHW] , see also 
Proposition 2.6 and Remark 2.7 below) and the character formulas for these modules 
can be obtained by applying the involution of the ring of symmetric functions with 
infinite variables, which sends the elementary symmetric functions to the complete 
symmetric functions, to the characters for the corresponding integrable modules over 
the respective Lie algebras (cf. [CK, CKW]). Remarkably, the u-homology groups 
of these modules are also dictated by those of the corresponding integrable modules 
[CK, CKW]. Recently, the correspondence between u-homology groups of integrable 
modules at positive levels and u-homology groups of unitarizable modules (at negative 
levels) over the respective Lie algebras can be elucidated in terms of the so called super 
duality [CWZ, CW], established in [BrS, CL, CLW]. So far there is no explanation 
of the similarity of these two different u-homology groups. Super duality gives a first 
conceptual explanation of this similarity [CLW, Theorem 4.13]. 

To the best of our knowledge, there is no other proof of Enright's formulas. In this 
paper, we give a proof of Enright's homology formulas for unitarizable modules by using 
Kostant's formulas and super duality. The u-homology formulas (see Theorem 4.4 be- 
low) for unitarizable modules over classical Lie algebras of infinite rank of types Ql{n), 
sp(2n) and so(2n) are obtained by combinatorial method. The proof involves relating 
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the combinatorial data of Kostant's formulas for integrable modules over corresponding 
Lie algebras, that are determined by the super duality, to the data of the Lie algebras 
under consideration. By applying the truncation functors (cf. [CLW, Section 3.4] to 
the u-homology formulas, see also Section 2.4 below), we recover the Enright's for- 
mula for three Hermitian symmetric pairs of classical types {SU {p, q), SU{p) x SU{q)), 
{Sp{2n), U (n)) and {SO*{2n), U (n)). However, for 50 (2n), our method can only recover 
partially Enright's formula for some unitarizable highest weight cases. 

The paper is organized as follows. In Section 2, we review and set up notations 
for the classical Lie algebras of finite and infinite rank. We describe the unitarizable 
highest weight modules considered in this paper. Combinatorial description of Weyl 
groups are also given in this section. In Section 3, we compare the actions of certain 
subsets of Weyl groups on certain numerical data associated with the highest weights. 
In Section 4, homology formulas for unitarizable modules over Lie algebras of infinite 
rank are proved. In Section 5, Enright's homology formulas are proved. 

We shall use the following notations throughout this article. The symbols Z, N, and 
stand for the sets of all, positive and non-negative integers, respectively. We set 
Z* := Z\{0}. For a partition A, we denote by A' the transpose partition of A. Finally 
all vector spaces, algebras, tensor products, et cetera, are over the field of complex 
numbers C. 

Acknowledgments. The second author is very grateful to Shun-Jen Cheng for nu- 
merous discussions and useful suggestions. He also thanks Weiqiang Wang for valuable 
suggestions. The first and second authors were partially supported by an NSC-grant 
and thank NCTS/South. The third author thanks NCTS/South for hospitality and 
support. 

2. Preliminaries 

2.1. Classical Lie algebras of infinite rank. In this subsection we review and fix 
notations on Lie algebras of interest in this paper. For details we refer to the references 
[K, W, CK, CLW]. 

2.1.1. The Lie algebra Oqo- Let C°° be the vector space over C with an ordered basis 
{ei\i € Z} so that an element in End(C°°) may be identified with a matrix (ajj) 
€ Z). Let Eij be the matrix with 1 at the i-th row and j-th column and zero 
elsewhere. Let doo denote the subalgebra of the Lie algebra End(C°°) spanned by 
Eij with i,j G Z. Denote by Ooo := doo © CK the central extension of doo by the 
one-dimensional center CK given by the 2-cocycle 

(2.1) TiA,B):=Tr{[J,A]B), 

where J = X]j<o En and Tr(C) is the trace of the matrix C. Observe that the cocycle 
r is a coboundary. Indeed, there is embedding from doo to Ooo defined hy A £ doo 
sending to A + Tv{JA)K (cf. [CLW, Section 2.5]). It is clear that ia(o^oo) is an ideal of 
Ooo and aoo is a direct sum of the ideals in(doo) and CK. Note that taiEu) = En + K 
(resp. Eii) for i < (resp. i > 1). 
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The Cartan subalgebra X^jg^ CEa © CK is denoted by l)a- By assigning degree to 
the Cartan subalgebra and setting degEij = j — i, Ooo is equipped with a Z-gradation 
cioo = ®kezi'^oo)k- This leads to the following triangular decomposition: 

Cloo = (aoo)+ ® (aoo)o © (cioo)-, 

where (aoo)± = ©fce±N('^oo)fc and (Ooo)o = ha- 

The set of simple coroots, simple roots and positive roots of Ooo are respectively 

n^f = { /3,^ := Eu - Ei+i,,+i + 6ioK\ieZ }, 

Ila = { ft := ei - ei+i | i G Z }, 

Aa = - ej M < i, «,i G Z}, 

where G f)* is determined by {ei,Ejj) = 5ij and {ei,K) = 0. We also let i?^ S {)* be 
defined by {i!}a,K) = 1 and {'&a,Ejj) = 0, for all j € Z. Let pa G ()„ be determined by 
{pa,Ejj) = —j, for all j € Z, and {pa,K) = 0, so that we have {pa,a^) = 1, for all 
i G Z. 

2.1.2. The Lie algebras Coo and t)oo- For 3 = c,t), let goo be the subalgebra of doo 
preserving the following bilinear form on C°°: 

if = 5, 

Let 000 = 0OO © CK be the central extension of 0oo determined by the restriction of the 
two-cocycle (2.1). Then 0oo has a natural Z-gradation and a triangular decomposition 
induced from aoo with (0oo)n = 0oo H (aoo)n, for n G Z. Similar to the the 
cocycle is a coboundary. Indeed, there are embeddings from 0oo to 0oo defined by 
^ G 0OO sending to A + Tr{JA)K [CLW, Section 2.5]. It is clear that i^{Q 00) is an ideal 
of 0OO and 0OO is a direct sum of the ideals tg(0oo) and CK. Note that '-g(-Ej) = Ei — K 
for z G N where 

E-i = Eii — Ei_i^i_i. 

Note that (0oo)o = SieN*^-^* © Cartan subalgebras, which will be denoted by 

[)g. We let et G f)* be defined by {ei,Ej) = 6ij for i,j G N and {ei,K) = 0. Then the 
set of positive roots of Cqo and doo are respectively 

A+ = { ± ei - ej , -2e, G N, i < j) }, 

A+ = { ±ei-ej GN,i <j)}. 

Set 

V \-Ei + K, for Coo, j-2ei, for Coo, 

\^-Ei-E2 + 2K, for t) 00, [-ei-e2, for Doo- 

The set of simple coroots and simple roots of 0oo are respectively 

U""^ ={a^,a^ = E,-Ei+i (i G N) }, 
Ilg = { ao, ai = ei- e^+i (f G N) }. 
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Let -dg G i)* defined by {^g,Ei) = for z € N and {^g,K) = r with r = 1 (resp. \) for 
3 = c (resp. 0). We also let € fi* be determined by 

[-J + 1, for0 = t), 
We have (/Og, a^) = 1 for i € N and g = c, t). 

2.1.3. Levi subalgebras. For 3 = a, c,c), let Ag := A+ U A~, where A~ = — A+. Then 

Ag is the set of roots of goo- Let A^^, := A^ n (Xljyo^'^j) ^^'^ ^g^n •= \ ^g^c- 
Denote by the root space corresponding to a G Ag. Set 

Then we have goo = © [g © u~ . The Lie algebras [g and goo share the same Cartan 
subalgebra f)g. Moreover, [g has a triangular decomposition induced from goo- For 
H E {)g, we denote respectively by L{qoo,p) and L{lg,fi) the irreducible highest weight 
goo-module and [g-module with highest weight p with respect to the triangular decom- 
positions. 

For a root a € Ag, g = a, c, t), define the refiection (Tq by 

(Ta{p^) := p - {p.,Q^)a, ^Gf]g. 

Here and after, denote the coroot of the root a. Let /a = Z and /g = N for g = c, 3. 
For j G /g U {0}, let (Tj = Let Wg be the subgroup of Aut(f)*) generated by the 
reflections aj with j € /g U {0}, i.e. is the Weyl group of goo- For each w G W^, 
^q{w) denote the length of w. We also define 

w o fi ■= w{fi + Pq) - Pq, /i G f)*,w; G VFg. 

Consider W^^ the subgroup of Wg generated by aj with j ^ 0. Let denote the set 
of the minimal length left coset representatives of Wg/Wg,o (cf. [V, Liu, Ku]). We have 
Wg = W^Wgfl. For k G Z+, set 

<,fc ■.= {weW^\£,{w) = k}. 
Finally, for g = a, c,c), let (-[-) be a bilinear form defined on subspace of fj* satisfying 
{ei\ej) = 6ij, (i9g|ei) = (ei|i9g) = (t?g|i9g) = for i,j G /g- 
Recall that Jq = Z and /g = N for g = c, t). 

2.2. Finite dimensional Lie algebras. For the rest of the paper, let g stand for 
a, c,c). We shall fix the following notations: 

:= 0, c := 5, d := c. 

Remark 2.1. For p = c,?), let g*^ and g^ be the Lie algebras defined in [CLW, Section 2] 
with m = 0. Then Coo = gS foo = g^, Coo — g*^ and Ooo — g''- Note that K send to —K 
for the isomorphisms Cqo — g*^ and Doo — g'' • 
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For m, n G N, the subalgebra of doo spanned by Eij with l — m < i,j < n, denoted by 
tm,nCii is isomorphic to the general linear algebra gl{m+n). The subalgebras (tn,nCi)nCoo 
and (tn,nfl) n doo are isomorphic to the symplectic Lie algebra sp(2n) and orthogonal 
Lie algebra so(2n), denoted by t^c and tnd respectively. We shall drop the subscript of 
t if there has no ambiguity. 

For g = a, c,t), the embeddings restricted to tg are also denoted by l^. Let 
denote the set of positive roots of tg with respect to the triangular decomposition 
induced from g^. We also let Atg = A+U-A+ and A+ „ = A±^nA+. Set fjtg = flgHtg, 
= n tg and [tg = % n tg. Note that tg and Itg share the same a Cartan subalgebra 
f)tg- Moreover, (tg has a triangular decomposition induced from tg. For /i G i]^-, we 
denote respectively by L(tg, ^) and L([tg,/i) the irreducible highest weight tg-module 
and Itg-niodule with highest weight /.i with respect to the triangular decompositions. 
For n G f)j-, L([tg,/i) is extended to an (Itg + u^)-module by letting act trivially. Let 
Ptg = kg + Define as usual the parabolic Verma module with highest weight by 

iV(tg,^) = Ind;^_L([tg,/.). 

The space t}^- is spanned by with 1 < i < n (resp. 1 — jtt, < i < n — 1) for g = c, t) 
(resp. o) and therefore i)^- can be regarded as a subspace of i}^. Note that [)j- is an 
invariant subspace of cTj for 1 < i < n (resp. l — m<i<n — l) for g = c or t) (resp. a). 
The restriction of these Uj to t)^-^ are also denoted by fij. Let Wig be the subgroup of 
Aut([)*-) generated by these cjjS. Then Wig is the Weyl group of tg. For each w £ Wig 
we let iig{w) denote the length of w. Consider VFtg,o the subgroup of W^tg generated by 
CTj with J 7^ 0. Let W^ denote the set of the minimal length representatives of the left 
coset space Wig/Wig^ (cf. [Liu, Ku]). For k G Z+, set :={w eW^^\ iig{w) = k}. 

We also define 

w o fi := w{fi + Pig) - Pig, ;U G {)tg, G VFtg- 

Finally, let pig denote the half sum of the positive roots. Then pig{h) = Pg{h) (resp. 
Pa{h) + \{n — m + 1)) for h G f)tg with g = c,ti (resp. g = o). 



2.3. Combinatorial descriptions of Weyl groups. In this section, we present com- 
binatorial descriptions of certain aspects of infinite Weyl groups Wg (cf. [BB]). Recall 
that Z* := Z\{0}. 
Define (^g G f^^ by 

^ ^ f-Ei<oei, if = a; 
^ lEieN^*, ifg = c,t). 

Every element a G f}* can be uniquely represented by + 9^0 with G C. 

For g = c, and i G N, we define e_j = — e^. It is easy to see by computing the actions 
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of Gi that the actions of Wq on f)* is given by 



(2.3) cr(^ iiei + gi9a) = ^{ii + Q)(^a{i) + ^'^^i^) + ^"^^ + if = a; 

i€Z i<0 i>0 

(2.4) cT(^eiei + g^?g) = X](^*-'?^^0'^))^-W + 9<^3'^)'^0 + ^^9' if0 = c>f> 

where a is a permutation of Z (i.e. a is a bijection on Z satisfying (7{j) = j for |j| ^ 0) 
for 3 = a and o" is a signed permutation of Z* (i.e. a is a bijection on Z* satisfying 
^(j) = J for |j| ^ and = for i G Z*) for 3 = c,c). Therefore a ^ a 

is a representation on Z and Z* for g = a and g = c,t), respectively. Moreover, they 
are faithful representations. It is clear that the image of in Aut(Z) is the set of 
permutations of Z and the image of (resp. W-^) in Aut(Z*) is the set of a signed 
(resp. even signed) permutations of Z*. A signed permutation a of Z* is called even 
signed permutation if |{i € N | a{i) < 0}| is a even number. We shall identify Wq with 
the image of in Aut(Z) (resp. Aut(Z*)) for = (resp. c, t)) for the rest of the 
paper. Note that for i G Z, ai{i) = i + 1, di{i + 1) = i and CFi{j) = j for all j 7^ i, i + 1. 
Also for g = c, 3 and z € N, o"i(i) = i + 1, (Tj(i + 1) = i and = j for all j ^ i^i + 1 
while a'o(l) = —1 (resp. —2), ao(2) = 2 (resp. —1), and (To(i) = J for all j > 3 for 
g = c (resp. t)). We shall use these representations for the rest of the paper and we 
shall simply write a{j) instead of o'(j). 

Recall that £g denote the length function on and denote the set of the minimal 
length left coset representatives of Wg/Wg^o- We have 



(2.5) W^^ 



(2.6) l,{a) 



{a e Wa I (T{i) < a{j) for i < j < and < i < j}, if g = a; 
G Ws\a{i) < a{j), for l<i<j}, if g = c,c) 

(see, e.g. [BB, Lemma 2.4.7, Proposition 8.1.4 and Proposition 8.2.4]) and for a G VFg , 

G Z X Z I i <j,a{i) > (T(i)}|, if g = a; 
|{(i,i) GNxN I i<j,a{-i)>a{j)}l if g = c; 
J{(i,i) gNxN I i <i,a(-i) >(t(j)}|, if g = t) 

(see, e.g. [BB, Corollary 1.5.2, Corollary 8.1.1 and Corollary 8.2.1]). 

Lemma 2.2. For a G with a{i) < for i < j, and a{i) > for i > j, define 
a G by 

'a{i) - 1, if i< j; 
1, if i = j + 1 and j is even; 

— 1 , if i = j + 1 and j is odd; 

\,a{i + ifi>j + 2. 

For each k > 0, the map from W^j^ to W^f^ sending a to a is a bijection. 

Proof. By (2.5), it is a bijection from to By (2.6), we have 4(o") = ^o(a) for 
a G W^. The lemma follows. □ 



a{i) 
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Let {'^jjjeN be a sequence of real numbers. Define := —S^i for i G N. For any 
sequence of strictly decreasing negative real numbers {CijigN and a G with g = c, 5, 
it is easy to see that {C(T{j)}jeN is a sequence of strictly decreasing real numbers. The 
following lemma follows from the definition of a. 

Lemma 2.3. Let {^i}igN be a sequence of strictly decreasing negative real numbers. 
Define ^^,^1 = for i € N and = 0. Then for all a G , we have 

M G N} U {0} = I i G N}, 

where a is defined in Lemma 2.2. 

2.4. Unitarizable highest weight modules. Recall that q stand for o, c,D, and o = 
a, c = c) and = c. In this subsection we classify the highest weights of irreducible 
unitarizable quasi-finite highest weight Qgo-modules with respect to the anti-linear anti- 
involution oj defined below. 

For a partition A = (Ai, A2, ■ ■ ■ )' t^e transpose partition of A is denoted by A' = 
(A'^, A2, • • • )• For = c, f, a partition A and d G C, define 

(2.7) A3(A,d):=^A:6, + d.?gGf);, A«(A, d) = J] A,6. - G 

Let T>{q) denote the set of pairs (A,d) with d G satisfying A'^^ < d if g = c; and 
A'l + A2 < d if g = c). For a pair of partitions A = (A~,A~'^) and d G C, define 
A»(A,d),A"(A,d) Gf)*by 

A°(A, d) = -Y, + j;(A+):e, + d^?„, 

A"(A, d) = -J2 + E ^t^^ - d^a- 

Let Ti{a) denote the set of pairs (A,d) satisfying d G Z+ and (A~)'^ + (A+)'^ < d. 

Let 6 be a Lie algebra equipped with an anti-linear anti-involution w, and let V be 
a 6-module. A Hermitian form ( • | • ) on 1/ is said to be contravariant if {av\v') = 
{v\oj{a)v'), for all a G f , z;' G V. A t-module equipped with a positive definite 
contravariant Hermitian form is called a unitarizable ^-module. Assume that t = (Bj^z^j 
(possibly dimtj = 00) is a Z-graded Lie algebra and is abelian. A graded 6- module 
M = (Bj^zMj is called quasi-finite if dim Mj < 00 for all j ^'L [KR]. 

Remark 2.4. Let V he a. highest weight goo-module with highest weight ^. Using the 
arguments as in [LZ, Section 4], we have V is quasi-finite if and only if ^ satisfies 
^{Eii) = (resp. ^{Ea) = 0) for \i\ » (resp. i » 0) for g = a (resp. c,d). Therefore 
every quasi-finite integrable highest weight goo-module is of the form L(goo, A0(A, d)) 
for some (A, d) G T){q). 

Now we consider the anti-linear anti-involution u on aoo defined by 

, , j Eji, for i, j < or i, j > 0; 
ujiEij) = < and ulK) = K. 

^ \ -Eji, for i > 0, j < or i < 0, j > 0, 
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For g = c, c), the restriction of the anti-linear anti-involution u on aoo to 0oo gives an 
anti- linear anti-involution on q^o, which will also be denoted by u. 

For d £ C and a pair of partitions A = (A~, A"*") with X^_^_i = A~_|_]^ = 0, let rta(A, d) 
be the element in f^^- determined by 

m n 

rta(A, d) = Y,i-d - K)^-^+l + Yl ^t^i- 
i=l i=l 

For d € C and a partition A satisfying A„_|_i = 0, let rtg(A, d) be the element in /i^- 
determined by 

'Er=i(Ai + ih, for0 = c, 
Ei=li>^^ + d)e^, for0 = t). 

Let T>i{Q) denote the subset of T>{q) consisting of elements in {X,d) satisfying A„+i = 
for = c, t) (resp. A^^^ = and A;;^^^ = for = a). 

Now we introduce the truncation functors [CLW, Section 3.4]. Let M = 0^ be 
a semisimple f)g-module such that is the weight space of M with weight f3. The 
truncation functor tr^^ is defined by sending M to 0^ M^, summed over Y17=i-m '^^i + 
Og (resp. Er=iCei + Og) for = a (resp. c,t)). For (A,d) e D{q), L{Q^X{\,d)) is 
a t0-module through the embedding defined in Section 2.2. ix^^{L{^^, {\, d))) is 
an irreducible t0-module and 

(2.8) ix,^^{L{Q^X{\d))) = L(t0,rtg(A,d)) 

for any partition A with A^+i = and = c,i) [CLW, Lemma 3.2]. The same result is 
also true for = a and pair of partitions A = (A~, A"*") with A^^^ = A~_,_^ = by using 
the same arguments as in [CLW]. The anti- linear anti-involution uj on 00^ induces an 
anti- linear anti-involution on f0, which will also be denoted by w. 

By cumbersome but straight forward computations, the following theorem is refor- 
mulated the Theorem 2.4 and some results of sections 7, 8, 9 in [EHW] in terms of 
partitions. 

Theorem 2.5. For = a, c, t), let ^ G f)^-. 

i. -L(ta, ^) is unitarizable with respect to uj if and only if^ = rt„(A, d)+k EIL-m+i ^* 
for some pair of partitions A = (A+, A~) with A^ = A+ = and d, /c G R safe- 
fying d>mln{{\~y^+n-l,{\+){ + m-l], or d £ Z and d > (A"); + (A+);. 
Moreover, A^(to, rtn(A, d) + kJ27=-m+i '^''"^ irreducible for pair of partitions 
A = (A"*", A~) with A~ = A+ = and d,k £ M. satisfying d > min{(A~)'i + n — 
l,(A+)'i + m- 1}. 

ii. -L(tc),^) is unitarizable with respect to uj if and only if £, = T^■^{X,d) for some 
partition A with A„, = and d G M satisfying d > n — 1 + X2, or d £ Z and 
d > X'l + \'2- Moreover, N {il) ,V ^{X, d)) are irreducible for partition X with 
A„ = and d > n — 1 + X^- 

iii. Assume that ^ £ fi^- with (,{En-i) = S,{En)- L{ic,^,) is unitarizable with respect 
to UJ if and only if i = rtc(A, d) for some partition X with A„_i = A„ = and 
d G M satisfying d > ^{X[ + n) — 1 if n — X[ is even; d > ^{X[ + n — 1) — 1 if 
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n — X'l is odd, or d ^ Ij and d > A'^. Moreover, N[\x.,T^{\,d)) are irreducible 
for partition A with A„_i = An = and d € M satisfying d > ^{X'l + n) — 1 if 
n — X'-^ is even; d > ^{X'l + n — 1) — \ if n — A'^ is odd. 

Proposition 2.6. For g = o, c, t), let L{q^,S^) be an irreducible quasi-finite highest 
weight Q^-module with highest weight Then L{q^,^) is unitarizable with respect to 
the anti-linear anti-involution u if and only if = A^(A, d) for some (A, d) G 

Proof. Let L{q^,(^) be a unitarizable irreducible quasi-finite highest weight 0oo-module. 
By Remark 2.4, satisfies ^(-Eii) = (resp. S,{Eii) = 0) for |z| > (resp. i > 0) for 
= a (resp. c,d). It is easy to see that d S M and £,{Eii) — ^(E'j+i^j+i) € (resp. 
£,[Eii) — ,^(£'j_|_i^j_|_i) G Z+) for all i (resp. i / 0) for g = c, £) (resp. a). This implies 
^ = A (A, d) for some partition A (resp. pair of partitions A = (A"*^, A~)) and d G M 
for g = c, D (resp. a). Now applying truncation functor to L{q^,E,) with n ^ d 
(resp. m,n ^ d) for g = c, c) (resp. a), ix^j^{L{Q^,$,)) is a unitarizable tg-module 
with respect to co. By Theorem 2.5 and (2.8), we have d ^ "L and (A, d) G T>i{Q). 
Hence ^ = A®(A,d) for some {\,d) G 23(g). Conversely, the irreducible highest weight 
goo-modules Li^^,K (A,d)) are modules appearing in the Howe dualities at negative 
levels described in [W]. These modules are unitarizable and quasi-finite. The proof is 
completed. □ 

Remark 2.7. The modules described in the proposition are modules appearing in the 
Howe dualities at negative levels described in [W] (cf. [LZ, Theorem 5.6, 5.8, 5.9]). 

3. Numerical data of the highest weights 

In this section, we shall provide combinatorial descriptions of A^(A,d) in terms of 
A0(A,(i). 

Definition 3.1. Let {ajjjgN and {6i}jGN be two strictly decreasing sequences of integers 
(resp. half integers). Then the sequences {ajjjgN and {6i}jGN are said to form a dual 
pair if Z (resp. ^ +Z) is the disjoint union of the two sequences {ajjjgpj and {— 6j}jgN. 

Define the function p on N by p[i) = —i for all i G N. The following lemma is well 
known (see e.g. [M, (1.7)]). 

Lemma 3.2. For any partition X, the sequences {Aj + /3(i)}jgN o-nd {A- + p{i) + IjigN 
form a dual pair. 

Recall that (j)^ = Yliim ^« 5 = ^5^- 

Lemma 3.3. For g = c,c) and {X,d) G T){q), let and {CijieN be two sequences 

determined by 



A%X,d)+p,-d{^g,K)ct>, = Y,C^^i + d^s^ 



A\X, d) + p-, + d{^„K)ck-, = Qe, - ^^^^^f. 
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Then {CijigN and {CiltgN form a dual pair. Moreover, Q < for i £ N and 7^ 5. In 
the case q = d, d < for i > 2, and Ci < (resp. = and > 0) for X[ < ^ (resp. = | 
and > 

Proof. By Lemma 3.2, and {CjjigN form a dual pair. It is clear that Q < 

for z € N and g = c. For g = t), we have A2 < | and hence Q < for i > 2. Also, 
(1 = a; - ^ < (resp. = and > 0) for A'^ < | (resp. = | and > ^). □ 

Lemma 3.4. For q = c,d and (A, d) E ^'(fl), let {CijieN cind {CilieN too sequences 
defined in Lemma 3.3. Define N{X,d) = {{i,j) € N x N | + Cj = 0, i,j G N}, 
J = {j en I (j. A;) ^ 7V(A, d), VA; € N}, S = {Ci M > 1} and S = {Ci I i G J }. 

i. For Q = c, we have S = S and Cd+i — 0- 

ii. For Q = d, we have 

S = § and Ci 7^ 7^ Ci for all i, if d is odd; 

S U {0} = S and Ci =0, if d is even and '^i ~ 

S = S and Ci = 0/or some i, if d is even and X'l ^ —. 

Proof. We shall only prove the case g = c). The proof of the other cases are similar and 
easier. For j > 2, we have Ci + Cj ^ A'^^ + A2 — d — j + 1 < —1 and hence Ci 7^ —Cj 
for j > 2. Since {CijigN and {CijieN form a dual pair, Ci 7^ ±Ci for J ^ 2 and Ci are 
negative for i > 2, we have Ci ^ S foi^ i ^ 2, and Ci S S for Ci 7^ 0. This implies 
S 5 S\{0}. For X G S, we have —x ^ S and hence —x G — S. Therefore S = S\{0}. By 
Lemma 3.2, S (resp. S) contains if and only if d is even and A'^^ = ^ (resp. A'^^ |). 
The proof is completed. □ 

Recall that (pa = — X]j<o ^i- 

Lemma 3.5. For (A,d) G D(a), let {Cijiez ctnd {Ci}i& be two sequences determined 
by 

A%\, d) + pa- d4>a = Y,iC^ - + d^a, 



A"(A, d) + p„ + dcPa = ^i^i - d^^- 



Define N{X,d) = {{i,j) G /„ x /„ | Q = Cj,i<0< j}, J+ = {j G N | {i,j) ^ 
N{X,d),yi < 0}, J__= {i e Z \ ^ Af(A,d),Vj_G N}, S+ = {0 \ i > 1}, 

S_ = {Ci I ^ < 0}, S-i- = {Ci I i G J+ } and S_ = {Ci | i G J_ }. Then we have 
S+ = — S_ and S_ = — S+. 

Proof Let S+ = {Ci | i G N} and S_ = {Ci | i<0}. By Lemma 3.2, we have 

(-S+)US+ = Z and (-§_)UB_=Z. 

For X G S+, we have x ^ by the definition of S+ and hence x G — S_. Therefore 
S+ C —§_. Now assume x G — S_. We have x ^ 23_. Since {Ci}iez is strictly increasing, 
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we have x ^ — S+ and hence x € 23+. Thus x € !B_|_\S_ = S+ and therefore — S_ C §_|_. 
Similarly, we have — S+ = S_. The proof is completed. □ 

We shall use the notations defined in Lemma 3.4 and Lemma 3.5 for the rest of the 
paper. By (2.3) and Lemma 3.5, we have (for (A,d) S 2)(a), a G Wa) 

(3.1) a~\A%\,d)+pa) = J^(Ci-l)e<x-i{i)+#a+dt?a = ^Ca{i)ei-^ei + dcl)^+di)a- 

ieZ i€Z ieZ 

By Lemma 3.4 and (2.4), we have (for (A, d) e D(g), a £ Wq and g = c, D) 

(3.2) a-i(A9(A, d) + P,) = Y1 Cw^i + d{^„K)(t>, + d^, + d^,. 

For T] belonging to the subspace of t)^ spanned by e^s and i9g, let [r/]"*" denote the 
unique A^^-dominant element in Wg.o-orbit of G f)i. The following two propositions 
are important for proving the main theorem in the next section. 

Proposition 3.6. Let {jijieN be the strictly increasing sequence with J = {ji \ i € N}. 
For (A, d) € '^{q) with q = c,d and a partition n with A^{fi,d) = a^^ o A^{X,d) for 
some a G W^f^, we have 

V{fi,d) + p^ + d{^„K)(t>^ 

Here appears only in the case Q = d and it is determined by = a (see Lemma 2.2 
and Lemma 2.3). 

Proof. In the proof, union means disjoint union. Let {^jjieN and {^jjieN be two se- 
quences determined by 



K\^,, d)+p-, + d(^g, K)<k-, = - ^^^^^ 

*6/„ 



Assume ^ §. We have {CjilieN = {CijiGN by Lemma 3.4. By Lemma 3.3, Lemma 3.4, 
and the fact that a acts on Z* as a signed permutation, we have 

{-Cw M e N} U {C,^^^^ I i e N} U {C, M G N\J} = Z (or i + Z). 

Since {Ci}ieN and form a dual pair, and {^^ | i G N} = {Co-(j) | « G N} by (3.2), 

we have {^j | z G N} = {Cj^^.j | i G N} U {C,^ \ i G N\J}. Therefore the proposition 

holds for this case since {^jjiGN is a decreasing sequence. 

The case of G § only occurs when g = t) with d = 0. We have {Cj, | i G N} = {^j | 
i G N}\{0}. Since cr acts on Z* as a signed permutation, by Lemma 2.3, we have 

I i G N} U {C.^^^^j I i G N} U {CJ i G N\J} = Z. 
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Now the proposition also follows in this case using the arguments above. □ 

Proposition 3.7. Let be the strictly decreasing sequence such that Jj^ = {ji \ 

z < 0} and J_ = {ki \ i € N}, and let J = J_ U J_|_. For (A, d) G 2?(a) and a partition 
with A°(/i, d) = o A°(A, d) for some a G W^f^, we have 

A°(/i,d) + + d(l)a = [ CjCi + Y(j^^i^^n - d'&a]^- 

i£Z\J i& 

Proof. In the proof, union means disjoint union. Let {^i}iGZ and {^jjiez be two se- 
quences determined by 

A°(/i, d) + + ^ ei - #a = ^ ii^i + d^a, 
A"(//, d) + Pa + d(l)a = ^liCi - dl^a- 

By Lemma 3.5, we have 

Z = (-S+) U (S+) U {CJ i G N\J+} = (-S+) U (-S„) U {CJ i G N\J+}. 

Therefore we have Z = {— Co-(i) | « G U {C^ | i G N\J+} because a acts as a 
permutation on Z. Since = C(T(j) for i G Z by (3.1) and ^^-(j) = — Cj^^.j for z G Z by 
Lemma 3.5, we have 

Z = I i G N} U M < 0} U {CJ i G N\J+} 

= {-e. I i G N} U {C,-^^^^ I i < 0} U {Ci M G N\J+}. 

Since {^ijieN and {■^jjiGN form a dual pair, we have {^j | i G N} = {Cj^^-j | i < 0}U{Cj | 

i G N\J+}. Similarly, we have {^^ | i < 0} = {C^^^.^ | i G N} U {C^ | i G (-Z+)\J„}. 

Therefore the proposition holds since {CijieN is a decreasing sequence and is 
an increasing sequence. □ 



4. U- -HOMOLOGY FORMULAS FOR 0oo-MODULES 

In this section we give a combinatorial proof of Enright's uT-homology formula [E] 

for the unitarizable highest weight g^^^-modules with highest weight A^{X,d). 

For a module V over Lie algebra S, let }ik{9',V) denote k-th homology group of 
S with coefficients in V. It is well known that the homology groups Hfc(ur;y) are 

[g-modules. The u^-homology of unitarizable highest weight modules are described by 
the following theorem which was obtained in [CK, Theorem 7.2] for = aoo and in 
[CKW, Theorem 6.5] for g^^ = Cqo, foo- The following theorem holds for more general 
situation by using the correspondence of homology group in the sense of super duality 
[CLW, Theorem 4.10] together with Kostant's formulas for integrable goo-modules (cf. 
[J, Ko, V, CK]). 
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Theorem 4.1. For (A, d) € T){q) with g = c, £) (resp. g = a), we have, as Ig-modules, 
Hfc(uZ; L(g^,r(A, d))) - L(% , A'(m, d)), 



where fi is summed over all partitions (resp. pairs of partitions) such that A^{fi,d) 



w o AB(A,d) for some w G Wgk- 



For ^ belonging the subspace of t}^ spanned by e^s and iJg, let ^'(^) = {a G Ai"^ | + 
/9g I a) = 0} and define <1>(^) to be the subset of A^^^ consisting of roots /3 satisfying the 
following conditions [E, DES]: 

il {(3\a) = for ah a £ ^(0; 
iii. f3 is short if ^(^) contains a long root. 

Let Wg(0 be the subgroup of Wg that is generated by the reflections Sjs with [3 S 
Define Ag(^) to be the subset of Ag consisting of the roots S Ag such that lies in 

For (A,d) € D(0), let Ag(A,(i) = Ag(A^(A,(i)) and W%(A,d) = W%(A^(A, d)). Then 
Ag(A, d) is an abstract root system and Vl^g(A, d) is the Weyl group of Ag(A, d) [E, EHW] 
(see also Lemma 4.2 below). Let A^ (A, d) = Ag(A, d)nA^ be the set of positive roots of 
Ag(A, d). Set iyg,o(A, d) = Wg{X, d) n W-^^q. Let W^{X, d) denote the set of the minimal 
length representatives of the left coset space Wq{X, d) /Wgfi{X, d) and let W^j^{X,d) be 
the subset of W^{X,d) consisting of elements a with i(x^d){(^) = ^) where ^(x^d) is the 
length function on Wg{X, d). 

For (A, d) G D(g), let J° = J U { j | Cj = 0} for g = c, t) and define 



T(A,d) 



'{ei - G A±; {i G J_, j G J+)}, for g = a; 

{-ei - ej G A±; (z < j, i,j G J°)}, for g = c; 

{-e, - ej G A+; {i < j, i,j G J)}, if J V ^ or f ^ Z,for g = t); 
^{-e^ - ej, -2ei G A±; (i < j, i,j G J)}, if J° = J and f G Z,for g = c). 



Lemma 4.2. For (A,d) G I'(g), we have 

Ag(A,d) 



' {ei - Ej G Ag; (z / j, i, j G J_ U J+)}, for g = a; 
{±{±ei - ej) G Ag; {i < j, i,j G J°)}, for g = c; 
{±(±e, - e,) G Ag; {i < j, i,j G J)}, if or f ^ Z, for g = t); 

^{±(±ei - ej),±2ei G Ag; (i < j, i,j G J)}, if J° = Jandf G Z,for Q = d. 



Proof. For (A, d) G ©(g), we have <1>(A®(A, d)) C T(A, d) by Lemma 3.4 and Lemma 3.5. 
Using the relations of the Weyl groups, it is easy to observe that T{X,d) C Ag(A, d). 
Now the lemma follows by using the relations of the Weyl groups again. 

□ 

Lemma 4.3. For {X,d) G T){q), there is a bijection from W^i^ to W^i^{X,d). 
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Proof. By Lemma 4.2, it is clear that W^/^ = W^^(A, d) for the cases g = a and = 

with j'' ^ J or I ^ Z. For the cases = c and = t) with = J and | € Z, the 
lemma follows from Lemma 4.2 and Lemma 2.2. □ 

Using Theorem 4.1, Proposition 3.6, Proposition 3.7, Lemma 4.3 and Lemma 2.3 
together with (2.3) and (2.4), we have the following theorem. 

Theorem 4.4. For {X,d) € 1^(0) and k € we have, as ig-modules, 

Hfc(ur;L(0^,A«(A,d))) - L{l^,[w-\A\X,d) + p^)]+-p^). 

Remark 4.5. There has the counterpart of the Theorem 4.11 in [CLW] for u^-cohomology 
in the sense of [Liu, Section 4]. The analogous statement is also true for = a. The 
formulas for u'^'-cohomology can be proved by the same argument as in the proof in 
[CLW]. Therefore, there is an analogue of Theorem 4.4 for u^-cohomology in the sense 
of [Liu]. The formulas for the cohomology can be proved by the same argument as in 
the proof of the theorem above. 

5. Homology formulas for unitarizable modules over finite dimensional 

Lie algebras 

In the section we shall give a new proof of Enright's homology formulas for uni- 
tarizable modules over classical Lie algebras corresponding to the three Hermitian 
symmetric pairs of classical types {SU{p,q), SU{p) x SU{q)), {Sp{n,'R),U{n)) and 
{SO*{2n),U{n)). 

For ^ belonging to f]^-, let ^{S,) = {a € ] (C + Ptg I a) = 0} and define <I>(^) to be 
the subset of consisting of roots (3 satisfying the following conditions [E, DES]: 

ii. (/31a) = for aU a G l'(^); 

iii. /3 is short if ^{(,) contains a long root. 

Let W^tg(^) be the subgroup of Wig that is generated by the reflections Sjs with /? G ^{0- 
Associated to Wtg(C)i let Atg(^) denote the subset of Atg consisting of the roots such 
that lies in VFtg(C)- We also let [^]"'" be the unique A^ ^-dominant element in the 
W^tg.o-orbit of ^. 

Assume that the irreducible module L(iQ, ^) is unitarizable with highest weight ^ € 
f)j-. Then Atg(^) is an abstract root system and TVtg(0 is the Weyl group of Atg(^) by [E, 
EHW]. Let A+(^) = Atg(0 n A+ be the set of positive roots of Atg(0- Set l^tg,o(0 = 
VFtg(^) n W^tg.o- Let W^{S,) denotes the set of the minimal length representatives of the 
left coset space VFtg(^)/VFtg^o(0 ^^^^ let ^tjfcCO be the subset of W^{(,) consisting of 
elements a with ^^(cr) = k, where is the length function on Wtg(^). 

Theorem 5.1. For = a, c or D, let L(t0, ^) be a unitarizable ^-module with highest 
weight € fi^^. Assume that ^ satisfies the assumption of the Case (Hi) of Theorem 2.5 
(of. Case (ii) of [EHW, Theorem 9.4]) for tg = so(2n). For k G Z+, we have, as 
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lig -modules, 

i7fcK-;L(t0,e)) = L([t5,[«;-i(e + ptg)]+-A0)- 

Proof. Since Hk{u^-; L{ta, fi + kY,1=-m+i^i)) = Hk{u--]L{m,^j))®L{k-^,kYTi=-rn+i^i) 
for alH > and /i G f)^-, it is sufficient to show all ^ with /c = appearing in the Case 
(i) of Theorem 2.5 when g = a. 

First we assume that ^ = rtg(A, d) with d ^ Z. Then we have Atg(^) = and 
-^(^0)0 = -^(tfl)?) by Theorem 2.5. Therefore L(fg, is a free Uj^-modules and hence 
i?fc(u^; L(t0, .^)) = L([tg,^) (resp. = 0) for A; = (resp. /c > 0). Thus the theorem 
holds for this case. 

Now we assume that ^ = Ti-^{\,d) for some (A,d) € 1*1(0) • By a direct calculation, 
we have Atg(^) = Ag(A,(i) n Atg. Recall that trj^(L(0oo, A^(A, d))) = L(t0, rtg(A, d)) 

for (A, d) G 1^1(0)- Since ir^|^(L(goo, A^(A, d))) = L(fg, rfg(A, d)) and the homology 
commutes with the truncation functor, we have 

iffc(u-;L(t0,e))=tr,f^(i^fc(Ug;L(0^,r(A,d)))). 

Note that Hk{u^]L(^^,K^[\,d))) with A; > decompose into the direct sum of ir- 
reducible [g-modules of the form L([g, A^(/i, d)) for some partition /i (resp. pair of 
partitions p. = (/i~,^+)) if = c,c) (resp. a) and trj^(L([g, A^(/x, d))) = L([tg, rtg(^, d)). 
Therefore, the theorem also holds for this case by Theorem 4.4. □ 

Remark 5.2. By Remark 4.5, Enright's cohomology formulas for unitarizable modules 
over classical Lie algebras with highest weights satisfying the assumption in the theorem 
above can be proved in the same manner as in above. 



References 

A. Bjorner and F. Brenti, Combinatorics of Coxeter groups, Graduate Texts in Mathematics, 
231. Springer, New York, 2005. 

J. Brundan and C. Stroppel, Highest Weight Categories Arising from Khovanov's Diagram 
Algebras IV: the general linear supergroup, arXiv:0907.2543. 

S.-J. Cheng and J.-H. Kwon, Howe duality and Kostant homology formula for infinite- 
dimensional Lie superalgebras, Int. Math. Res. Not. (2008) Art. ID rnn 085, 52 pp. 
S.-J. Cheng, J.-H. Kwon and W. Wang, Kostant homology formulas for oscillator modules 
of Lie superalgebras. Adv. Math. (2010), doi:10.1016/j.aim.2010.01.002. 

S.-J. Cheng and N. Lam, Irreducible characters of the general linear superalgebra and super 
duality, Commun. Math. Phys. 298 (2010), 645-672. 

S.-J. Cheng, N. Lam and W. Wang, Super duality and irreducible characters of ortho- 
symplectic Lie superalgebras, Invent, math., DOI 10.1007/s00222-010-0277-4. 
S.-J. Cheng and W. Wang, Brundan-Kazhdan-Lusztig and Super Duality Conjectures, Publ. 
Res. Inst. Math. Sci. 44 (2008), 1219-1272. 

S.-J. Cheng, W. Wang and R. B. Zhang, Super duality and Kazhdan-Lusztig polynomials. 
Trans. Amer. Math. Soc. 360 (2008) 5883-5924. 

M. Davidson; T. J. Enright and R. Stanke, Differential operators and highest weight repre- 
sentations. Mem. Amer. Math. Soc. 94 (1991), no. 455, iv-l-102 pp. 



[BB] 

[BrS] 

[CK] 

[CKW] 

[CL] 

[CLW] 

[CW] 

[CWZ] 

[DES] 



16 



PO-YI HUANG, NGAU LAM, AND TZE-MING TO 



[E] T. J. Enright, Analogues of Kostant's u-cohomology formula for unitary highest weight mod- 

ules, J. Reine Angew. Math. 392 (1988) 27-36. 

[EHW] T. J. Enright, R. Howe and N. R. Wallach, A classification of unitary highest weight modules, 
Representation Theory of Reductive Groups, Boston 1983, 97-143. 

[HI] R. Howe, Remarks on classical invariant theory, Trans. AMS 313 (1989), 539-570. 

[H2] R. Howe, Perspectives on invariant Theory: Schur Duality, Multiplicity-free Actions and 

Beyond, The Schur Lectures, Israel Math. Conf. Proc. 8, Tel Aviv (1992), 1-182. 

[J] E. Jurisich, An Exposition of Generalized Kac-Moody Algebras. Lie algebras and their rep- 

resentations (Seoul, 1995), 121-159, Contemp. Math. 194, Amer. Math. Soc, Providence, 
RI, 1996. 

[K] V. Kac, Infinite-dimensional Lie algebras, Third edition, Cambridge University Press, Cam- 

bridge, 1990. 

[KR] V. Kac and A. Radul, Quasi-finte highest weight modules over the Lie algebra of differential 
operators on the circle, Commun. Math. Phys. 157 (1993) 429-457. 

[Ko] B. Kostant, Lie Algebra Cohomology and the generalized Borel-Weil Theorem, Ann. Math. 74 

(1961), 329-387. 

[Ku] S. Kumar, Kac-Moody groups, their flag varieties and representation theory. Progress in 

Mathematics, 204. Birkhauser Boston, Inc., Boston, MA, 2002. 

[LZ] N. Lam and R. B. Zhang, Quasi-finite modules for Lie superalgebras of infinite rank. Trans. 

Amer. Math. Soc. 358 (2006), 403-439. 

[Liu] L. Liu, Kostant's Formula for Kac-Moody Lie algebras, J. Algebra 149 (1992), 155-178. 

[M] I. G. Macdonald, Symmetric functions and Hall polynomials, Oxford Math. Monogr., Claren- 

don Press, Oxford, 1995. 

[V] D. A. Vogan, Representation of Real Reductive Lie Groups, Progress in mathematics; vol. 

15, Birkhuser, 1981. 

[W] W. Wang, Duality m infinite dimensional Fock representations, Commun. Contem. Math. 1 

(1999) 155-199. 

Department of Mathematics, National Cheng Kung University, Tainan, Taiwan 70101 
E-mail address: pyhuangOmail .ncku. edu. tw 

Department of Mathematics, National Cheng Kung University, Tainan, Taiwan 70101 
E-mail address: nlam@mail . ncku . edu . tw 

Department of Mathematics, National Changhua University of Education, Changhua, 
Taiwan, 500 

E-mail address: matotm@cc.ncue.edu.tw 



